Image stitching technology is a technology for stitching several images with overlapping sections to produce a large seamless high-resolution image. The field of view of a camera can be increased by image stitching. Compared with methods that use a lens with a larger field of view to increase the field of view, the method for image stitching results in less distortion.
Conventional methods for two-dimensional images stitching, for example, SIFT (Scale Invariant Feature Transform) based stitching algorithms, are computationally intensive and are prone to errors in complex scenes, resulting in poor quality of stitched images. Moreover, during image stitching, ghosting may easily appear in a transitional zone due to the difference between the calibrated depth and the actual depth. For another example, depth-image-based stitching methods employ a plurality of visible light cameras to acquire image information, and perform stitching based on depth information obtained according to the principle of stereoscopic vision among cameras. Because these methods require matching of feature points in real-time, the real-time effect of stitching is affected.
With the technology for obtaining depth information becoming more and more mature, a method for image stitching in combination with depth information is proposed in the prior art, including: searching for an overlapping area between a target image and an image to be stitched, obtaining the depth information of the overlapping area through a depth camera, obtaining an amount of parallax based on the depth information, and mapping pixels of the overlapping area to the target image. Compared with conventional methods for two-dimensional image stitching, the method for image stitching in combination with depth information described above can be applied to stitch images in more complex scenes. In addition, an additional depth camera is added on the basis of a plurality of visible light cameras to obtain the depth information of the overlapping area, which can solve the problem of ghosting in the conventional methods for two-dimensional image stitching. However, this method has the following problems. First, the prior art needs to perform pre-calculation to obtain an overlapping area, obtain a depth image of the overlapping area through a depth camera, and align depth information of pixels in the overlapping area with visible light information, resulting in a less efficient stitching process. Second, the prior art needs to identify an overlapping area, and map the pixels in the overlapping area and those in an extended area in two different ways. There may be an abrupt change in a transitional zone, resulting in an undesirable fusion of a transitional segment between the overlapping area and the extended area. Furthermore, in related processes of image stitching based on depth information, the depth information is not used in the mapping process to transform pixels in the extended area. When pixels in an image to be stitched are mapped to a target image, due to the change of viewpoints, visible pixels that can be observed in the image to be stitched may be invisible in the stitched target image, thus an occlusion of foreground and background cannot be handled very well.
There have been no effective solutions proposed yet to solve the problem of a low stitching efficiency in the image stitching process above.